[pull] master from cube-js:master - #646
Merged
Merged
Conversation
#11425) * refactor(tesseract): move symbol-mutating logic into symbols/transforms module Symbols are immutable values; every derived-copy operation now lives in planner/symbols/transforms as a plain function instead of a method on the symbol types: - unroll_rolling (was MeasureSymbol::new_unrolling) - patch_measure (was MeasureSymbol::new_patched) - into_multiplied / regular_in_multiplied (was into_multiplied / convert_multiplied_to_regular; kind-level logic stays on MeasureKind) - apply_static_filter_to_symbol and friends (moved from symbols/common, absorbing the replace_case methods) - substitute_by_name (extracted from FullKeyAggregateMeasures::render) - strip_join_prefix (was MemberSymbol::with_stripped_join_prefix plus per-type strip_join_prefix methods) Symbol struct fields are pub(super) so transforms rebuild them via full struct literals: adding a field fails to compile until every transform classifies it, the same guarantee symbol_deps! gives for traversal. * refactor(tesseract): replace the ignored-timezone render map with a symbol property A time dimension read from a pre-aggregation rollup or from a rolling window input CTE carries an already timezone-converted value. This was tracked in SqlNodesFactory as a full-name set consulted at render time β matching members by name during SQL generation, invisible in the plan. The property now lives on the symbol itself: TimeDimensionSymbol gets an ignore_timezone flag, and the selects that read such sources are built from schemas whose time dimensions (and their occurrences inside filter trees) carry the flag. TimeDimensionNode reads it from the symbol; the factory set, its setter and both fill sites are gone. New transform layers, by input level: - planner/symbols/transforms: ignore_timezone_for (recursive symbol rewrite) and map_filter_symbols/map_filter_item_symbols β a generic filter-tree symbol walker that static-filter application now shares. - logical_plan/transforms: ignore_timezone_in_schema β schema-wide lift of the symbol transform. * refactor(tesseract): replace the count-approx-as-state render flag with a measure kind form The mergeable HLL state of count_distinct_approx was a render-time boolean: SqlNodesFactory.count_approx_as_state, threaded from the physical build (pre-aggregation builds) and from the multi-stage evaluation context (leaves under a rolling window), consulted by the final-measure nodes during SQL generation. The state form is now a MeasureKind variant, mirroring MultipliedCount: AggregatedState renders as AggregateWrap::CountDistinctApproxState (hll_init; hll_merge when read back from a rollup). It materializes at logical planning via the transforms::measures_as_state tree rewrite β in QueryProperties finalize for pre-aggregation builds and in the multi-stage leaf CTE under an aggregating stage β so the logical plan itself shows that a leaf produces a state. Removed: the factory flag and both final-node fields, PushDownBuilderContext.render_measure_as_state, EvaluationContext.measure_as_state, and the pre_aggregation_query parameter of PhysicalPlanBuilder::build. Every decision match over MeasureKind, AggregateWrap and their inner aggregation-type enums is now exhaustive β a future form variant fails to compile at each decision point instead of falling through a wildcard arm. One deliberate behavior change: a count_distinct_approx measure reachable only through a dimension dependency tree (e.g. a subquery dimension) used to render as an HLL state inside the dimension subquery under the flag; it now renders as a final value, since a dimension value cannot be a state blob. * chore(tesseract): drop dead sql nodes, document member-identity equality cube_calc_groups.rs and original_sql_pre_aggregation.rs were not wired into the module tree and matched a MemberSymbol variant that does not exist. MemberSymbol's PartialEq compares member identity (full_name + variant); the doc comment now states that symbol content does not participate, so derived forms of a member compare equal to the original and this equality must not be used to distinguish them. * test(tesseract): guard ungrouped rendering combined with masking, multiplied measures, order by, rolling Result snapshots capture the semantics these combinations must keep: a masked measure stays masked in an ungrouped query (including a mask whose SQL has row-level dependencies), a multiplied count keeps its distinct form, ORDER BY of an unselected measure sorts by the row-level value, and a rolling count-distinct leaf emits the raw distinct key. None of these interactions were covered before. * refactor(tesseract): replace the ungrouped render flags with a measure render modifier The row-level rendering of measures was two render-time booleans: SqlNodesFactory.ungrouped / ungrouped_measure, set per select by the physical processors, branching the final-measure chain and switching MaskedSqlNode into its ungrouped mode. The form now lives on the symbol: MeasureSymbol.render_modifier β Option<MeasureRenderModifier> with Ungrouped (raw row value: measure subqueries, ungrouped multi-stage leaves) and UngroupedQueryValue (row value in an ungrouped query; count-likes render a not-null indicator). It materializes when each select is built, from the same inputs the flags used: the pushdown context's measure_for_ungrouped takes precedence over the select's ungrouped modifier, and ORDER BY symbols of unselected measures are included. The factory builds all three measure chains and MeasureRenderModifierSqlNode routes per-measure. Masking keeps its positional asymmetry explicitly: only the node wrapping the final measure chain applies row-level mask semantics (deferring dependency-carrying masks to the evaluate-position node, which always masks with grouped semantics), now derived from the measure's modifier instead of select-level flags. Removed: both factory flags and setters, MaskedSqlNode::new_ungrouped, and the dead MeasureSymbol.is_splitted_source field. Known deviation: a measure referenced inside another member's mask filter renders through the unmasked root without a modifier (final aggregation) where the old select-level flags applied; this corner moves with the masking rework. * refactor(tesseract): replace rolling and multi-stage window render flags with measure render modifiers The last per-select render decisions living on SqlNodesFactory move to the measure symbols of the selects they describe: - RollingMerge replaces the rolling_window flag: the rolling-window select stamps it on its measures, and the dispatcher routes them to the RollingWindowNode chain (window-partial merge by kind), which the factory now builds unconditionally. - MultiStageRank / MultiStageWindow { partition } replace the multi_stage_rank / multi_stage_window partition strings: the partition travels as member symbols and is rendered through the regular chain, resolving to the same qualified columns via the select's render references. The processor keeps the alias-existence guard and stamps a schema copy used for both projections and ORDER BY. The rank/window nodes keep their chain position and kind checks but dispatch by the measure's modifier. The modifier enum carries data now, so it is Clone (not Copy), and masking derives row-grain semantics only from the two ungrouped variants β rolling and windowed measures keep grouped mask semantics as before. SqlNodesFactory is left with reference maps, cube aliases, time shifts and group-by member names only. * refactor(tesseract): make render-modifier stamping precise, assert form applicability at render MeasureRenderModifier::applies_to is the single authority for which measures take a form: stamping consults it (a modifier lands only on compatible measures instead of every measure of the select), and the rank/window render nodes assert it, failing loudly on an incompatible combination instead of silently falling through. The dispatcher arm for rank/window measures is an internal error β they must be intercepted by their dedicated nodes. Renames for accuracy: Ungrouped β RawValue (a row-level value re-aggregated by an enclosing select), UngroupedQueryValue β UngroupedFinal (the final row-level output of an ungrouped query), ignore_timezone β tz_converted_at_source (the fact, not an instruction; it composes with future derived forms). Documented the dual meaning of a None modifier (no decision yet / final aggregation β coincident because stamping only fills None) and the transform rebuild-style rule (full struct literal when a transform decides per field, clone-and-mutate for single-field stamps). * refactor(tesseract): tighten render-form handling per review - assert RollingMerge applicability in the render-modifier dispatcher - drop the render modifier when unrolling a rolling measure - mark tz-converted-at-source across all schema members, so embedded granularity references carry the mark too; unit test on the transform - pin conditional dependency-carrying mask behavior in ungrouped queries * refactor(tesseract): close render-form coverage gaps found in review Stamping a render form now reaches every carrier the removed query-wide flags covered: WHERE filters beside HAVING, and measures embedded in other schema members' expression trees. A state form merges like the aggregation it stores, and render marks survive deriving another form of the same time dimension. - stamp the measure render modifier on WHERE filters and on all schema members; reuse the schema transform in the multiplied subquery - merge AggregatedState like Aggregated in the rolling-window node, drop its now-dead non-cumulative fallback - carry render marks through TimeDimension reference/granularity forms - MeasureRenderModifier::ensure_applies_to replaces three copies of the render-side assertion; share the PARTITION BY rendering - unit-test applies_to, ensure_applies_to and the state form; cover the embedded-member cases of both schema transforms - skip rebuilding order-by items already stamped in the schema; skip the tz rewrite when no name matches - correct docstrings that outlived the fields they described
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? π Please sponsor : )